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EXECUTIVE  SUMMARY 


A  retrospective  analysis  was  undertaken  to  better  understand  the  performance  trade-offs  involved  in 
reducing  the  sensor  count  in  the  Spectral-Based  Volume  Sensor  Component  Prototype  developed  as  part 
of  the  Advance  Volume  Sensor  Project.  Using  a  portion  of  the  Volume  Sensor  Test  Series  4  (VS4)  data, 
superior  performance  was  retrospectively  demonstrated  for  several  potential  new  configurations  than  was 
originally  observed  during  the  live  VS4  demonstration.  Additionally,  it  was  possible  to  maintain  a  large 
fraction  of  the  performance  while  utilizing  fewer  sensor  elements  than  the  original  SBVS  Component 
Prototype  configuration. 

The  SBVS  Component  Prototype  demonstrated  as  part  of  VS4  was  comprised  of  sensor  suites  with  five 
optical  sensors  (center  wavelengths  of  5900  A  (Na),  7665  A  (K),  10500  A  (NIR),  solar-blind  UV  (UV), 
and  4.3  pm  (IR))  and  five  EVENT  algorithms  (EVENT,  PDSMOKE,  FIRE,  FIREFOV,  and  WELDING) 
generating  alarm  events  for  distribution  to  the  overall  Volume  Sensor  Prototype  (VSP).  A  two-element 
configuration  (UV/IR)  shows  improved  performance  over  the  as-tested  VS4  configuration.  Additionally, 
the  probability  of  detection  (Pd)  for  the  WELDING  EVENT  was  improved  from  0.93  to  1.00  through 
optimization.  The  performance  of  the  FIREFOV  EVENT,  which  did  not  alarm  for  any  of  the  selected 
test  scenarios  during  the  live  demonstration,  was  further  improved  to  yield  18  out  of  60  detections. 

Increasing  the  number  of  sensor  elements  in  the  configuration  leads  to  increased  Pd  for  the  FIRE  EVENT 
with  increasing  probability  of  false  alarm  (Pfa).  The  K/UV/IR  combination  offers  little  improvement  in 
the  FIRE  Event  Pd  (0.82  vs.  0.80)  while  doubling  the  number  of  false  alarms.  This  combination  is  not 
recommended.  The  NIR/UV/IR  combination  offers  significant  improvement  in  the  FIRE  EVENT  Pd 
(0.95  vs.  0.80)  while  only  adding  5  additional  false  alarms  over  the  UV/IR  combination.  Especially 
interesting  was  that  the  FIRE  EVENT  false  alarms  generated  were  moved  from  the  nuisance  and  welding 
test  scenario  classes  to  the  cutting  and  grinding  class.  Hot  work  of  these  types  are  not  typically  conducted 
without  significant  preparation  and  system-wide  notification  in  a  shipboard  environment  and  any  DCA 
system  would  most  likely  be  secured  or  operating  in  a  special  mode  to  handle  this  type  of  work.  This 
reduces  the  potential  severity  of  the  generated  false  alarms.  Using  all  four  elements  achieves  an  almost 
perfect  Pd  for  the  FIRE  EVENT  (0.98  or  0.97  vs.  0.80)  while  adding  only  5-7  additional  false  alarms, 
depending  on  the  threshold  setting  for  the  primary  SBVS  detection  parameter,  SumN.  The  performance 
statistics  for  the  WELDING  and  FIRE  FOV  EVENTS  were  unchanged  in  any  of  these  configurations. 

An  example  of  potential  inter-component  data  fusion  is  discussed  for  using  the  LWVD  Component 
luminosity  value  in  lieu  of  the  NIR  sensor  element.  One  could  potentially  use  the  LWVD  luminosity  data 
stream  in  lieu  of  the  NIR  data  stream  to  produce  an  effective  3 -component  SBVS  system  with  only  the 
UV  and  IR  sensor  elements.  Or  an  effective  4-component  system  could  be  produced  with  only  the  K, 

UV,  and  IR  sensor  elements.  This  would  require  that  the  LWVD  data  stream  exhibit  similar  sensitivity 
and  dynamic  range  to  that  of  the  K  photodiode.  Data  analysis  along  these  lines  is  currently  ongoing. 

On  the  basis  of  these  analyses,  the  current  recommendation  for  the  SBVS  Component  in  the  AiOVS  being 
developed  by  Vibro-Meter,  Inc.  (VMI)  currently  under  ONR  sponsorship  is  either  a  UV/IR  or  a 
NIR/UV/IR  configuration,  depending  on  the  data  fusion  systems’  tolerance  for  increased  Pfa  with 
increased  Pd. 
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1.  INTRODUCTION 

The  Advanced  Volume  Sensor  Project  was  one  element  of  the  Office  of  Naval  Research’s  Future  Naval 
Capabilities  program,  Advanced  Damage  Countermeasures.  This  program  sought  to  develop  and 
demonstrate  improved  damage  control  (DC)  capabilities  to  help  ensure  that  the  recoverability 
performance  goals  for  new  ship  programs,  such  as  the  CVN21  and  the  DD(X)  families  of  ships,  could  be 
met  with  the  specified  manning  levels  and  damage  control  systems.  Using  a  multi-sensory  approach,  the 
Naval  Research  Laboratory  (NRL)  is  developing  new  detection  capabilities  for  DC  in  the  shipboard 
environment.  Conventional  surveillance  cameras,  which  are  currently  being  incorporated  into  new  ship 
designs,  provide  the  basis  for  the  Advanced  Volume  Sensor  (VS)  project.  Video  Image  Detection  (VID) 
is  an  emerging  technology  for  the  remote  detection  of  events  within  the  camera’s  field  of  view  (FOV)  by 
applying  image  analysis,  or  machine  vision,  techniques  to  the  video  image.  Optical  sensor  systems 
sensitive  to  radiation  outside  the  visible  spectrum  and  acoustic  sensors  are  also  being  developed  in 
combination  with  the  VID  technologies  to  produce  an  overall  sensor  system  that  is  able  to  provide  a  broad 
range  of  situational  awareness  for  the  sensor’s  entire  field  of  view.  The  use  of  remote  sensing  techniques 
removes  the  constraint  of  typical  smoke  and  fire  detection  systems  that  rely  on  diffusion  of  gases, 
particles,  or  heat  to  the  detector.  A  Volume  Sensor  Prototype  (VSP)  was  developed  at  NRL  to  provide  an 
affordable,  real-time,  robust,  and  remote  detection  sensor  system  that  provides  detection  and  classification 
of  damage  control  conditions  such  as  fire,  explosions,  pipe  ruptures,  and  compartment  flooding.  The  VSP 
generates  alarm  notifications  for  action  by  the  Damage  Control  Assistant  and  other  available  damage 
control  systems  based  on  the  detected  event.  The  VSP  was  successfully  demonstrated  in  simulated 
shipboard  conditions  in  several  test  series  conducted  on  the  ex-USS  Shadwell  in  Mobile,  AL. 

The  design  goal  of  the  Spectral-Based  Volume  Sensor  (SBVS)  Component  is  to  detect  fire,  smoke,  and 
other  hazardous  conditions  using  optical  methods  outside  the  visible  region  of  the  electromagnetic 
spectrum.  The  sensors  developed  within  the  SBVS  Component  are  intended  to  be  used  in  conjunction 
with  and  to  augment  the  performance  of  the  core  VID  technology  of  the  VSP.  The  VID  systems  are 
generally  better  at  identifying  smoke  than  fire,  so  a  primary  goal  of  the  SBVS  Component  is  to  provide 
better  detection  for  flame  and  fire.  An  important  constraint  in  the  VS  project  is  that  the  eventual  system 
must  be  affordable  with  a  target  unit  cost  well  below  $1000.  This  precludes  the  use  of  more  obvious 
solutions  such  as  mid-infrared  (IR)  cameras  because  the  per-unit  price  is  too  high  (>  $10,000  per  unit). 
Two  avenues  have  been  pursued  in  parallel  within  the  SBVS  Component.  One  approach  employs  long 
wavelength  video  detection  (LWVD),  emphasizing  the  benefits  of  spatial  resolution  and  near  infrared 
imaging  afforded  by  readily  available,  inexpensive  video  cameras.  Descriptions  and  results  of  the  LWVD 
system  are  provided  in  other  reports  [1,2]  and  in  a  patent  [3].  The  second  avenue  utilizes  single-element 
detectors  operating  in  several  narrow  spectral  regions  from  the  IR  to  the  ultraviolet  (UV)  that  correspond 
to  the  wavelengths  of  several  peak  flame  emissions. 

Initial  laboratory  and  shipboard  testing  of  the  VSP  in  2004  -  2005  was  extremely  successful.  In  2008,  the 
U.S.  Congress  allocated  funds  to  Vibro-Meter,  Inc.  (VMI)  through  the  Office  of  Naval  Research  to 
further  the  development  of  the  VSP  towards  a  commercially  available  product.  The  Volume  Sensor 
Detection  Suite  prototype  development  effort  is  a  joint  effort  between  VMI  and  NRL  to  develop  a 
commercially-viable  VSP  unit  in  a  single  package.  The  design  and  development  of  the  All-In-One 
Volume  Sensor  (AiOVS)  involves  the  evaluation  of  each  sensor  element  in  terms  of  added  value  to  the 
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performance  of  the  AiOVS  and  the  associated  cost  to  incoiporate  that  element.  This  report  documents 
these  analyses  for  the  SBVS  Component  Prototype  sensor  elements  and  detection  algorithms. 

The  organization  of  this  report  is  as  follows.  Sections  2  and  3  briefly  discuss  SBVS  Component 
Prototype  hardware  and  event  detection  algorithms,  respectively,  as  they  have  been  tested  in  Volume 
Sensor  Test  Series  3-5  [4-6].  Section  4  provides  a  brief  description  of  the  VS4  Test  Series,  the  data 
from  which  provided  the  basis  of  the  analysis  described  in  this  report.  Additional  information  can  be 
found  in  Reference  5.  Section  5  discusses  the  development  of  the  All-In-One  Volume  Sensor  (AiOVS) 
Prototype  and  the  supporting  additional  analysis  of  the  SBVS  event  algorithms  associated  with  this  effort. 
Sections  5  and  6  give  the  final,  optimized  results  for  the  SBVS  event  algorithms  based  on  the  VS4  test 
data  and  recommendations  for  the  development  of  the  AiOVS  Prototype.  Section  7  contains  the 
references  cited  in  this  document.  Appendix  A  contains  the  complete  definitions  of  the  SBVS  EVENT 
algorithms.  Appendix  B  documents  the  finalized  calibration  factors  for  the  10  existing  VSP  units. 
Appendix  C  documents  the  parameter  values  for  the  various  SBVS  configurations  discussed  in  this  report. 


2.  SBVS  COMPONENT  PROTOTYPE  HARDWARE 

The  SBVS  Component  Prototype  is  described  in  detail  elsewhere  [7]  and  is  only  discussed  briefly  here. 
Each  SBVS  Component  Prototype  sensor  suite  is  composed  of  two  units,  the  VIS/IR  unit  and  the  UV 
unit.  A  typical  installation  of  a  SBVS  Component  Prototype  sensor  suite  for  the  VS5  Test  Series  is 
shown  in  Figure  2-1.  The  VIS/IR  unit  contains  three  Si  photodiodes  (PDs)  with  interference  filters 
centered  at  5900  (Na),  7665  (K),  and  10500  (NIR)  A  (bottom  unit,  starting  from  the  right  in  Figure  2-1), 
each  with  a  full-width,  half- maximum  (FWHM)  band  width  of  ~10  nm.  Each  unit  has  a  mid-IR  (IR) 
detector  installed  with  a  central  operating  wavelength  of  4.3  pm  (bottom  unit,  left-hand  element).  Several 
of  the  units  have  a  second  IR  detector  with  a  central  operating  wavelength  of  2.7  pm  (2.7  pm  +  4.3  pm 
for  sensor  suite  #53).  The  data  from  the  second  IR  detector  are  not  currently  used  by  any  algorithm  and, 
where  present,  are  recorded  only  for  future  research  and  development.  The  UV  units  (upper  unit  in 
Figure  2-1)  are  designed  around  a  standard  UV-only  OFD  (Vibrometer,  Inc.).  The  OmniGuard  860 
Optical  Flame  Detector  (Vibrometer,  Inc.)  used  in  the  original  SBVS  Testbed  contained  the  same  UV 
sensor  unit.  As  of  present,  nine  pairs  of  VIS/IR  and  UV  units  and  a  single  AiOVS  mock-up  unit  have 
been  fabricated.  As  outlined  in  a  previous  report  [8],  a  distributed-architecture  data  acquisition  system 
was  designed  and  implemented  for  the  SBVS  Component  Prototype  of  the  VS  Prototype  using  the 
FieldPoint  line  (National  Instruments)  of  data  acquisition  equipment. 
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Figure  2-1  -  Typical  installation  of  the  SBVS  Component  Prototype  as 
tested  in  VS5  Test  Series.  The  UV  unit  is  positioned  above  the  VIS/IR 
unit.  See  the  text  for  a  description  of  the  individual  elements. 


3.  SBVS  COMPONENT  PROTOTYPE  ALGORITHMS 

Event  detection  algorithms  for  five  events  were  implemented  for  the  real-time  use  of  the  SBVS 
Component  Prototype.  The  development  of  these  algorithms  is  presented  in  Reference  9.  These  events 
are:  EVENT,  PDSMOKE,  FIRE,  FIREFOV,  and  WELDING.  The  EVENT  was  conceived  as  a  generic 
trigger,  indicating  that  some,  currently  unclassifiable  event  is  occurring  in  the  FOV  of  the  sensor.  The 
PDSMOKE  event  makes  use  of  long-time-scale  deviations  observed  in  the  5900  A  channel  data  that  were 
not  correlated  with  flaming  events  to  detect  and  classify  smoke  within  the  sensor  FOV.  The  algorithms 
for  FIRE  and  FIRE  FOV  detection  compare  the  measured  channel  data  “spectrum,”  or  the  pattern  of 
channel  values  for  the  five  sensors  to  an  empirically  determined  spectrum  for  a  fully  involved  flaming  fire 
in  the  sensor  FOV  for  the  FIRE  FOV  event,  or  to  a  more  relaxed  spectrum  for  the  FIRE  event.  An 
algorithm  for  the  positive  detection  of  one  type  of  nuisance,  arc  welding  was  also  included.  To  reduce  the 
algorithm  sensitivity  to  transient  signals,  a  persistence  criterion  of  five  seconds  was  applied  to  the 
algorithm  outputs  (25  seconds  for  the  PDSMOKE  algorithm).  All  raw  channel  data  were  recorded  locally 
on  one  of  the  SBVS  Component  data  acquisition  computers.  Baseline-subtracted  and  normalized  sensor 
channel  data  and  algorithm  outputs  were  forwarded  to  the  VSP  Fusion  Machines  (FM)  using  the  VSCS 
communications  protocol.  Based  on  analysis  of  the  VS3  test  series  data,  two  changes  were  made  to  the 
SBVS  Component  Prototype  algorithms  of  Reference  9  to  improve  performance.  First,  the  PDSMOKE 
algorithm  was  modified  to  correctly  allow  for  both  positive  and  negative  deviations  in  the  5900  A  channel 
data.  Also,  individual  calibrations  were  implemented  for  the  SBVS  hardware  in  each  sensor  suite  to 
allow  for  unit-to-unit  variations. 

Appendix  A  gives  a  complete  listing  of  the  SBVS  EVENT  parameters  and  the  EVENT  algorithm 
descriptions.  See  Reference  9  for  further  details.  The  FIRE  EVENT  algorithm  definition  is  given  below. 
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Fire:  IF  (Sum  N  >=  0.0825)  and  (7665A  >=  0.015)  and 

(10500A  >=  0.015)  and  (RefIR/UV  >=  1)  Then 
FIRE  =  TRUE 

Else 

FIRE  =  FALSE. 

As  part  of  the  original  SBVS  algorithm  development,  a  principal  component  analysis  (PCA)  was 
conducted  on  all  of  the  data  channels  in  the  SBVS  TestBed.  A  single  principal  component  was  identified, 
labeled  SumN,  which  is  defined  as  the  sum  of  the  scaled  signals  from  the  four  principal  sensor  elements 
in  the  SBVS  Component  Prototype:  the  K  and  NIR  PDs,  the  UV,  and  the  IR  sensors.  Procedurally, 

SumN  is  defined  as: 

SumN  =  7665A  +  10500A  +  (0.1  *  RefIR)  +  UV 

In  Reference  9,  the  definition  of  SumN  included  the  5900  A  (Na)  PD  data  values.  In  subsequent 
algorithm  iterations,  the  Na  PD  data  was  completely  compartmentalized  into  the  PDSMOKE  EVENT 
algorithm  and  not  used  in  either  the  determination  of  SumN  or  in  the  FIRE,  FIRE  FOV,  or  WELDING 
EVENT  definitions.  From  the  definition  of  the  FIRE  EVENT,  several  thresholds  are  apparent  and  these 
are  the  parameters  that  will  be  varied  in  this  analysis  for  each  potential  configuration  of  the  new  AiOVS. 

4.  VOLUME  SENSOR  4  TEST  SERIES 

4.1.  GENERAL  INFORMATION 

The  objective  of  the  VS4  test  series  [5]  was  to  evaluate  prototype  sensor  suites  and  alarm  algorithms 
onboard  the  ex-USS  Shadwell  in  preparation  for  demonstrating  VSP  systems  in  FY05.  In  particular,  the 
tests  were  designed  to  assess  the  developmental  progress  of  the  VSP  system  since  the  Test  Series  3 
evaluation  in  July  2004  and  to  expand  the  database  of  scenarios  and  sensor  measurements.  These  tests 
were  conducted  October  18-29,  2004. 

Full-scale  experiments  were  conducted  aboard  the  ex-USS  Shadwell  in  Mobile,  AL  [10].  This  test  series 
consisted  of  small  fires,  adjacent  space  fires,  various  nuisance  sources,  and  pipe  ruptures  that  challenged 
the  detection  systems.  Two  VSPs  comprised  of  three  prototype  sensor  suites,  one  of  the  evaluated  VIDS, 
and  containing  newly-developed  data  fusion  algorithms  were  installed  for  the  test  series.  The 
performance  of  the  VSPs  and  the  VID  systems  were  compared  to  the  response  of  commercial  off  the  shelf 
(COTS)  smoke  detection  technologies. 

4.2.  SELECTED  TEST  SCENARIOS 

The  tests  were  conducted  in  and  around  the  mock  magazine  on  the  3rd  deck  of  the  ex-USS  Shadwell.  The 
test  matrix  for  the  VS4  Test  Series  consisted  of  one  hundred  (100)  test  scenarios.  A  variety  of  fire, 
nuisance,  pipe  rupture,  and  gas  release  sources  were  created  to  expose  the  VSPs  and  spot-type  detectors  to 
a  range  of  potential  shipboard  scenarios.  Small  fires  were  used  to  challenge  the  detection  systems  and 
provide  performance  results  for  early  detection.  A  number  of  the  nuisance  sources  involved  people 
moving  about  the  space.  Pipe  ruptures  were  simulated  with  a  range  of  flow  rates  and  leakage  areas  to 
challenge  the  VSP.  Further  information  on  the  VS4  test  matrix  can  be  found  in  Reference  5.  For  the 
purposes  of  SBVS  algorithm  development,  52  exemplar  VS4  test  scenarios  were  selected  which 
represented  unique,  single  events  relevant  to  the  SBVS.  There  were  three  SBVS  sensor  suites  installed  in 
the  test  space  with  differing  FOVs  in  the  compartment,  for  a  total  of  156  data  sets.  The  test  scenarios 
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were  broken  down  into  four  classes:  20  flaming,  5  welding  tests,  8  cutting  and  grinding,  and  19  nuisance 
test  scenarios.  The  specific  tests  are  identified  in  Table  4-1  -  Table  4-4. 

Table  4-1  -  Test  Series  VS4  Selected  Flaming  Test  Scenarios 


Test 

SBVS  Root  Filename 

Test  Description 

VS4-001 

Oct  182004  133159 

Flaming  Cardboard  Boxes  with  Polystyrene  pellets 

VS4-006 

Oct  192004  093503 

Flaming  Trash  Can 

VS4-009 

Oct  192004  130003 

Flaming  Shipping  Supplies 

VS4-015 

Octl92004  160403 

Flaming  IPA  Spill  Fire  /  Trash  bag 

VS4-018 

0ct202004  102359 

Flaming  Cardboard  Boxes  with  Polystyrene  pellets 

VS4-022 

Oct2 02004_  131459 

Flaming  Trash  Can 

VS4-026 

0ct202004  163000 

Flaming  Cardboard  Boxes  with  Polystyrene  pellets 

VS4-027 

Oct2 12004  082558 

Flaming  Shipping  Supplies 

VS4-032 

Oct212004  112458 

Flaming  IPA  Spill  Fire  /  Trash  Bag 

VS4-038 

Oct2 12004  154958 

Flaming  Trash  Can 

VS4-043 

Oct222004  093358 

Flaming  Shipping  Supplies 

VS4-044 

Oct222004  103758 

Flaming  IPA  Spill  Fire  /  Trash  Bag 

VS4-050 

Oct252004  084500 

Flaming  Shipping  Supplies 

VS4-053 

Oct252004  100600 

Flaming  IPA  Spill  Fire  /  Trash  Bag 

VS4-058 

Oct252004  124359 

Flaming  Cardboard  Boxes  with  Polystyrene  pellets 

VS4-068 

Oct262004  100059 

Flaming  IPA  Spill  Fire  /  Trash  Bag 

VS4-070 

Oct262004  120759 

Flaming  Cardboard  Boxes  with  Polystyrene  pellets 

VS4-082 

Oct272004  092458 

Flaming  Cardboard  Boxes  with  Polystyrene  pellets 

VS4-091 

Oct272004  160158 

Hot  metal  surface  -  IPA  spill  under  slanted  cab  door 

VS4-096 

Oct282004_l  04258 

Flaming  trash  can  -  Camera  4  tilted  up 

Table  4-2  -  Test  Series  VS4  Selected  Welding  Test  Scenarios 


Test 

SBVS  Root  Filename 

Test  Description 

VS4-002 

Oct  182004  143359 

Welding 

VS4-008 

0ctl92004_121902 

Welding  (140  A) 

VS4-083 

Oct272004  100558 

Welding 

VS4-090 

Oct272004  152358 

Welding  preceded  by  no,  normal,  and  high  ventilation 

VS4-092 

Oct272004  163357 

TIG  welding  stainless  steel 

Table  4-3  -  Test  Series  VS4  Selected  Cutting  and  Grind  Test  Scenarios 


Test 

SBVS  Root  Filename 

Test  Description 

VS4-005 

Oct  1 92004_08400 1 

Torch  Cut  Steel 

VS4-014 

Oct  192004  153802 

Grinding  Painted  Steel 

VS4-025 

0ct202004  155331 

Torch  Cut  Steel 

VS4-035 

Oct2 12004  135258 

Grinding  Painted  Steel 

VS4-056 

Oct252004  111359 

Grinding  Painted  Steel 

VS4-077 

Oct262004  150959 

Grinding  Painted  Steel 

VS4-085 

Oct2  72004  121658 

Grinding  Painted  Steel 

VS4-089 

Oct272004  145858 

Torch  Cut  Steel 
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Table  4-4  -  Test  Series  VS4  Selected  Nuisance  Test  Scenarios 


Test 

SBVS  Root  Filename 

Test  Description 

VS4-004 

Oct  182004  163659 

VHF  Radio  /  People  working 

VS4-010 

Oct  192004  134803 

Waving  materials 

VS4-012 

Octl92004  150203 

Spilling  Metal  Bolts 

VS4-013 

Oct  192004  151903 

AM/FM  Radio  /  Cassette  player 

VS4-016 

0ct202004  083159 

Engine  Exhaust 

VS4-019 

0ct202004  110859 

TV  /  People  working  in  space 

VS4-020 

0ct202004  113659 

TV  with  video 

VS4-024 

0ct202004  151700 

Normal  Toasting 

VS4-029 

Oct2 1 2004_ 100257 

People  working  in  space  -  clean  up  pipe  mpture 

VS4-034 

Oct2 12004  132657 

Heat  gun  /  space  heater  /  fan 

VS4-052 

Oct252004  095000 

People  working  in  space  -  clean  up  pipe  rupture 

VS4-055 

Oct252004  105300 

People  working  in  space  -  clean  up  pipe  mpture 

VS4-064 

Oct262004  083159 

Engine  Exhaust 

VS4-066 

Oct262004  092359 

Aerosol 

VS4-067 

Oct262004  094359 

Flash  Photography 

VS4-081 

Oct272004  091058 

Flash  Photography  with  four  people 

VS4-098 

Oct282004  124158 

Space  heater 

VS4-099 

Oct282004  130058 

Toaster 

VS4-100 

Oct2  82004  131258 

Space  heater 

5.  SB  VS  ALGORITHM  DEVELOPMENT 

A  final  demonstration  was  planned  for  the  Advanced  Damage  Countermeasures  FNC  in  the  Fall  of  2005. 
The  post-VS5  Volume  Sensor  Prototypes  were  to  be  included  in  this  demonstration  as  an  integrated  part 
of  an  overall  damage  control  system.  While  the  VSPs  had  previously  demonstrated  significant  added 
performance  for  detection  and  classification  of  damage  control  events,  the  prototype  nature  of  the  system 
did  not  allow  one  to  easily  envision  the  potential  final  products.  An  example  of  a  complete  VSP  sensor 
suite  installation  is  shown  in  Figure  5-1.  NRL  and  VMI  determined  that  it  would  be  advantageous  to 
have  available  a  mock-up  of  what  a  production  unit  might  look  like,  the  more  functional  the  better.  This 
was  also  considered  an  opportunity  to  review  the  added  value  of  each  of  the  5  remaining  SB  VS  sensor 
elements  and  determine  which  ones  were  required  to  maintain  performance.  Due  to  the  events 
surrounding  Hurricane  Katrina,  the  final  demonstration  was  never  held,  but  the  AiOVS  mock-up  has  been 
useful  at  various  meetings  and  presentations  as  a  talking  point  that  can  be  held  in  one’s  hand.  During  a 
magazine  test  conducted  in  the  Winter  of  2008,  as  part  of  a  new  program,  the  VSPs  and  the  AiOVS 
mock-up  were  installed  for  demonstration. 
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IR  and  UV 
sensors  (SBVS) 


Microphone 
(AC  ST) 


Video 
camera  (VI D) 


Filtered  video 
camera  (LWVD) 


Figure  5-1  -  Volume  Sensor  Prototype  Sensor  Suite  including 
VIDS,  ACST,  SBVS,  and  LWVD  components. 
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5.1.  ALL-IN-ONE  PROTOTYPE 

Based  on  preliminary  evaluations  (See  Section  5.3.2  for  further  discussion),  the  AiOVS  mockup  shown  in 
Figure  5-2  was  constructed  by  VMI  with  two  Si-CCD  cameras  (one  color  CCTV  and  one  LWVD  filtered 
B&W  camera),  the  IR  and  UV  sensors  from  the  SBVS  Component,  and  a  microphone. 


Figure  5-2  -  All-in-One  Volume  Sensor  mock-up  constructed  by  VMI  in  2005. 

As  part  of  ONR’s  Volume  Sensor  Detection  Suite  prototype  development  effort,  the  preliminary  analyses 
that  lead  to  the  down-selection  of  the  SBVS  sensor  components  to  only  the  IR  and  UV  sensors  was  to  be 
reviewed  in  greater  detail.  The  remainder  of  this  report  will  discuss  these  analyses. 

5.2.  DEVELOPMENT  TOOLING 

To  partially  automate  the  data  analysis  process,  a  software  package,  SBVS  Replay,  was  developed  in- 
house  at  NRL  to  reprocess  previously  collected  data  files  while  allowing  sensor  calibration  and  algorithm 
threshold  parameters  to  be  varied.  A  screenshot  of  the  program  operating  is  shown  in  Figure  5-3.  The 
operator  can  select  one  or  more  archival  data  files  from  Test  Series  VS3,  VS4,  and  VS5  /  DD(X).  The 
label  DD(X)  refers  to  several  test  series  conducted  in  2005  for  which  the  VSPs  were  run  in  the  VS5 
configuration  along  side  the  main  demonstration.  The  tool  is  currently  limited  to  only  being  able  to 
process  data  from  one  test  series  at  a  time  because  the  data  file  formats  were  updated  prior  to  each  of 
these  test  series.  The  program  automatically  detects  the  test  series  type  from  the  file  format  and  loads  the 
appropriate  calibration  factors  for  each  sensor  unit.  For  VS3  and  VS4  data  files,  the  calibration  factors 
are  displayed  in  the  upper  left  of  the  screen  and  are  user-editable  from  within  the  program.  Due  to  the 
large  number  of  parameters  for  VS5  /  DD(X),  the  calibration  factors  are  stored  in  the  system  registry  and 
can  be  viewed  /  updated  there.  The  operator  is  then  able  to  select  which  of  the  five  sensor  elements  will 
be  considered  in  the  reprocessing  (lower  left  of  screen).  The  right-hand  side  of  the  screen  is  devoted  to 
the  algorithm  thresholds  and  parameters.  The  persistence  in  seconds,  or  duration  of  an  event,  required  to 
trigger  an  event  are  displayed  in  the  middle  left  of  the  screen.  Once  the  operator  has  configured  the  run  as 
desired,  the  selected  data  files  are  reprocessed  using  the  new  parameters  and  a  composite  output  file  is 
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generated  for  all  files  processed.  This  program  allows  the  operator  to  systematically  vary  a  parameter  and 
observe  the  results  easily. 


Figure  5-3  -  Screenshot  of  the  SBVSReplay  tool. 


5.3.  ANALYSIS  RESULTS 

In  support  of  ONR’s  Volume  Sensor  Detection  Suite  prototype  development  effort,  a  detailed 
examination  of  the  relative  added  value  of  each  sensor  component  was  evaluated  with  the  goal  of 
reducing  the  SB  VS  sensor  count  from  the  current  5  to  2  or  3  sensors  in  a  rigorous  and  defensible  manner 
while  retaining  or  even  improving  performance.  The  original  SBVS  algorithms  were  developed  from  the 
SBVS  Testbed  sensor  data  collected  during  the  VS2  Test  Series  [9].  Given  the  continued  evolution  of  the 
SBVS  Component  Prototype  since  that  time,  a  fresh  look  was  warranted.  A  subset  of  the  VS4  Test  Series 
data  archive  was  selected  for  analysis  as  described  in  Section  4.  One  caveat  that  should  be  kept  in  mind  is 
that  SBVS  calibration  procedures  for  the  individual  units  were  under  development  throughout  this  period. 
All  new  analyses  discussed  in  this  document  use  the  final  VS5  calibration  values  as  listed  in  Appendix  B. 
The  VS5  recommended  algorithm  formulations  were  used  as  an  initial  starting  point  as  well. 
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5.3.1.  VS4  TEST  SERIES  AS-TESTED  RESULTS 

The  ‘As-Tested’  results  for  the  SBVS  Component  Prototype  for  the  52  selected  test  cases  are  summarized 
in  Table  5-1  and  shown  in  Figure  5-4.  Using  a  color-coding  scheme  which  will  be  used  for  the  remainder 
of  this  discussion,  results  which  are  considered  beneficial  to  performance  are  color-coded  green  and 
results  which  are  considered  detrimental  to  performance  are  color-coded  red.  The  numbers  in  the  lower 
right  of  each  cell  represent  the  number  of  test  scenarios  (tests  *  #  of  sensor  suites  (3)).  The  numbers  in 
the  upper  left  of  each  cell  represent  the  number  of  EVENT  alarms  found  by  the  reprocessing  with  the 
SBVSReplay  software.  As  tested,  the  SBVS  Component  Prototype  detected  41  of  the  60  flaming  test 
scenarios,  all  via  the  FIRE  EVENT.  The  WELDING  EVENT  detected  14  of  the  15  welding  test 
scenarios.  The  FIRE  FOV  EVENT  failed  to  alarm  for  any  test  scenario,  the  FIRE  EVENT  false  alarmed 
for  7,  and  the  WELDING  EVENT  false  alarmed  for  5  of  the  24  cutting  and  grinding  test  scenarios.  The 
PD  SMOKE  EVENT  alarmed  as  given  in  the  Table. 

Table  5-1  -  SBVS  Component  Prototype  ‘As-Tested’  algorithm  results 
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FIRE 

PDSMOKE 

FIRE_FOV 

WELDING 


Test  Scenarios 

Figure  5-4  -  SBVS  Component  Prototype  ‘As-Tested’  algorithm  results  by 
test  scenario  type 


In  Figure  5-5,  the  same  results  as  presented  in  Table  5-1  are  shown,  classified  now  for  comparison  to  the 
overall  VSP  performance  results.  The  FIRE  and  FIREFOV  EVENT  outputs  are  combined  such  that  an 
alarm  from  either  EVENT  will  produce  a  FLAME  alarm.  The  two  EVENTS  are  treated  equally.  The 
PDSMOKE  and  WELDING  EVENTS  directly  map  to  the  SMOKE  and  WELDING  alarms. 


To  simulate  an  overall  system  response,  the  RESPOND  alarm  is  a  composite  of  the  other  three  alarms 
such  that  if  either  the  FLAME  or  SMOKE  alarms  are  generated  and  the  WELDING  alarm  is  not  active,  a 
RESPOND  alarm  is  generated.  For  the  VS4  SBVS  Component  Prototype,  the  RESPOND  results  are 
given  for  the  four  test  scenario  classes  in  Table  5-2. 
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Figure  5-5  -  SBVS  Component  Prototype  ‘As-Tested’  response  results  by 
test  scenario  class 


Table  5-2  -  SBVS  Component  Prototype  ‘As-Tested’  response  results  by  test  scenario  class 


Actual  VS4  Performance 

FLAME 

SMOKE 

WELDING 

RESPOND 

Flaming  Tests 

41 

25 

0 

46 

Welding  Tests 

0 

8 

14 

0 

Cutting  and  Grinding  Tests 

7 

0 

5 

6 

Nuisance  Tests 

0 

5 

0 

5 

For  comparison,  the  VS4  As-Tested  response  results  for  both  VSPs  are  given.  Each  VSP  used  the  same 
SBVS,  LWVD,  and  acoustics  (ACST)  data  streams  and  one  of  the  VIDS  data  streams:  either  the  Fastcom 
SFA  (FM1)  or  axonX  Signifire  (FM2).  The  FM1  and  FM2  response  results  are  given  in  Figure  5-6  and 
Figure  5-7,  respectively,  and  in  Table  5-3  and  Table  5-4,  respectively.  For  both  VSPs,  the  overall  system 
demonstrated  superior  detection  results  as  compared  to  the  SBVS  Component  Prototype  alone,  as  would 
be  expected.  Flowever,  both  VSPs  were  significantly  more  vulnerable  to  false  alarms  from  the  nuisance 
tests  than  the  SBVS  Component  Prototype  alone. 
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Figure  5-6  -  VSP  FM1  response  results  by  test  scenario  class 


Table  5-3  -  VSP  FM1  response  results  by  test  scenario  class 


FM1  -  Fastcom 

Flame 

Smoke 

Pipe 

Rupture 

Respond 

Flaming  Tests 

51 

57 

0 

60 

Welding  Tests 

0 

0 

0 

0 

Cutting  and  Grinding  Tests 

6 

3 

0 

9 

Nuisance  Tests 

0 

18 

6 

18 

14 
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Figure  5-7  -  VSP  FM2  response  results  by  test  scenario  class 


Table  5-4  -  VSP  FM2  response  results  by  test  scenario  class 


FM2  -  axonX 

Flame 

Smoke 

Pipe 

Rupture 

Respond 

Flaming  Tests 

51 

54 

0 

60 

Welding  Tests 

0 

0 

0 

0 

Cutting  and  Grinding  Tests 

6 

0 

0 

6 

Nuisance  Tests 

0 

12 

6 

15 

5.3.2.  INITIAL  OPTIMIZATION  (2005) 

An  initial  assessment  of  the  potential  for  down-selecting  SBVS  sensor  elements  for  inclusion  in  the 
AiOVS  mock-up  was  conducted  in  2005  prior  to  the  construction  of  the  AiOVS  mock-up  by  VMI.  A 
two-element  configuration  was  selected  using  the  UV  and  IR  sensor  elements  from  the  SBVS  Component 
Prototype.  The  results  are  shown  in  Figure  5-8  and  Figure  5-9  and  Table  5-5  and  Table  5-6.  The  FIRE 
EVENT  correctly  detected  48  of  the  60  flaming  test  scenarios  with  only  4  false  alarms.  One  false  alarm 
was  a  special  welding  test  scenario  (VS4-08)  where  a  high-power  welder  was  used.  This  test  scenario 
was  not  repeated.  The  remaining  three  false  alarms  were  from  the  nuisance  test  scenarios  and  from  ‘hot 
work’  nuisances  such  as  heat  guns  and  space  heaters  (VS4-34,  -98,  and  -100).  In  each  of  the  four  test 
cases,  only  one  of  the  three  SBVS  Component  Prototypes  in  the  space  alarmed.  This  allows  for  potential 
suppression  of  the  false  alarm,  if  appropriate,  at  the  data  fusion  level  of  the  VSP  using  compartment-level 
situational  awareness.  For  the  FIREFOV  EVENT,  optimization  led  to  detections  in  14  of  the  60  flaming 
test  scenarios  with  no  false  alarms  in  the  other  classes.  The  FIRE  FOV  algorithm  is  designed  to  be  an 
indication  of  a  larger  fire  than  might  be  detected  by  the  FIRE  EVENT,  so  it  is  not  possible  to  know  a 
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priori  what  the  correct  value  should  be  with  the  data  available.  The  WELDING  EVENT  was  optimized 
and  had  a  Pd  of  1.00  for  the  welding  test  scenario  class  with  5  false  alarms  in  the  cutting  and  grinding  test 
scenario  class.  Based  on  these  analyses,  the  two-component  configuration  was  suggested  for  the  AiOVS 
mock-up.  The  PDSMOKE  columns  are  grayed  out  to  reflect  the  fact  that  without  the  Na  PD  data  stream, 
the  PDSMOKE  EVENT  cannot  generate  an  EVENT. 


FIRE 

PDSMOKE 

FIRE_FOV 

WELDING 


Test  Scenarios 

Figure  5-8  -  SBVS  Component  Prototype  2005  UV/IR  configuration 
algorithm  results  by  test  scenario  class 


Table  5-5  -  SBVS  Component  Prototype  2005  UV/IR  configuration 
algorithm  results  by  test  scenario  class 


FIRE 

PDSMOKE 

FIREFOV 

WELDING 

Flaming 

48 

60 

0 

60 

14 

60 

0 

60 

Welding 

0 

15 

0 

15 

15 

15 

Cutting  and 
Grinding 

0 

24 

0 

24 

0 

24 

Nuisances 

0 

57 

0 

57 

0 

57 
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Figure  5-9  -  SBVS  Component  Prototype  2005  UV/IR  configuration 
response  results  by  test  scenario  class 


Table  5-6  -  SBVS  Component  Prototype  2005  UV/IR  configuration  response  results  by 

test  scenario  class 


Initial  Optimization  -  UV  /  MIR 

Only 

FLAME 

SMOKE 

WELDING 

RESPOND 

Flaming  Tests 

48 

0 

0 

48 

Welding  Tests 

1 

0 

15 

0 

Cutting  and  Grinding  Tests 

0 

0 

5 

0 

Nuisance  Tests 

3 

0 

0 

3 

5.3.3.  FATE  OF  THE  PDSMOKE  EVENT 

Only  the  PDSMOKE  EVENT  incorporates  the  5900  A  data  stream  (other  than  the  generic  EVENT 
EVENT)  in  the  SBVS  Component  Prototype.  The  main  focus  of  the  SBVS  Component  Prototype  has 
always  been  on  flaming  sources,  to  which  the  VIDS  are  not  very  sensitive.  Therefore,  it  was  determined 
that  the  modest  performance  of  the  PDSMOKE  EVENT,  as  compared  to  the  VIDS,  shown  in  the  previous 
section  is  not  worth  the  additional  cost  of  retaining  the  Na  PD  in  the  AiOVS.  The  5900  A  data  stream 
was  not  considered  further  in  this  analysis.  As  the  PDSMOKE  EVENT  is  decoupled  from  the  other 
EVENTs,  it  would  be  a  fairly  straight  forward  activity  to  restore  and  optimize  the  PDSMOKE  capability 
at  a  later  date,  if  so  desired. 
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5.3.4.  FIRE  AND  FIRE  FOV  EVENTS 

A  focused  reprocessing  and  analysis  of  the  performance  of  the  FIRE  and  FIREFOV  EVENTS  was 
undertaken.  The  VS4  As-Tested  results  show  that  the  measured  performance  of  these  EVENTs  was  not 
acceptable,  with  no  FIRE  FOV  EVENT  detections  and  detection  of  only  41  of  60  flaming  test  scenarios 
by  the  FIRE  EVENT.  Additionally,  improvement  of  the  false  alarm  rate  for  the  FIRE  EVENT  (7  of  96) 
was  desirable.  Each  available  parameter  of  each  EVENT  was  systematically  varied  and  evaluated  using  a 
threshold  curve.  An  example  is  shown  in  Figure  5-10.  The  context  of  the  example  will  be  explained  later 
in  this  section.  In  this  example,  the  FIRE  EVENT  for  a  particular  sensor  configuration  is  being  evaluated. 
The  probability  of  detection  (Pd,  the  fraction  of  flaming  test  scenarios  correctly  classified),  the  probability 
of  correct  classification  (Pcorr,  the  fraction  of  flaming  test  scenarios  correctly  classified  as  FIRE  and  the 
number  of  non-flaming  test  scenarios  not  classified  as  FIRE),  and  the  probability  of  false  alarm  (Pfa,  the 
fraction  of  non-flaming  test  scenarios  incorrectly  classified  as  FIRE)  are  plotted  as  a  function  of  the  FIRE 
EVENT  SumN  threshold  value.  The  blue  vertical  line  indicates  both  the  initial  stating  point  and  the 
optimal  operating  point,  which  are  the  same  in  this  case.  Increased  SumN  threshold  leads  to  a  reduced  Pd 
while  decreased  SumN  threshold  leads  to  an  increase  in  Pfa  (and  corresponding  decrease  in  Pcorr). 


Figure  5-10  -  Example  threshold  curve  for  the  optimization  of  the  FIRE  EVENT 
SumN  threshold  for  the  K/UV/IR  3-element  combination 

Using  this  analysis  scheme,  four  different  potential  configurations  for  the  SBVS  portion  of  the  AiOVS 
were  studied  and  their  performance  results  for  the  same  data  set  were  determined.  Four  potential  data 
streams  are  available,  the  7665  A  Si  PD  (K),  the  10500  A  Si  PD  (NIR),  the  UV  gas  discharge  tube  (UV), 
and  the  4.3  pm  IR  (IR)  detector.  The  four  configurations  are:  a)  UV/IR,  b)  K/UV/IR,  c)  NIR/UV/IR,  and 
d)  K/NIR/UV/IR.  The  evaluation  of  each  EVENT  (the  FIRE  and  FIRE  FOV  EVENTs,  and  the 
WELDING  EVENT  in  the  next  section)  for  a  data  point  is  made  independently.  This  allows  the 
performance  of  each  EVENT  to  be  optimized  independently. 


18 


Steinhurst  and  Owrutskv 


The  results  for  the  four  configurations  are  given  in  Table  5-7  through  Table  5-16  and  Figure  5-11  through 
Figure  5-20.  In  the  case  of  the  4-element  configuration,  results  for  two  different  SumN  thresholds  for  the 
FIRE  EVENT  are  presented  as  a  balance  between  Pd  and  Pfa  considerations. 


Table  5-7  -  UV/IR  configuration  algorithm  results 


FIRE 

PDSMOKE 

FIREFOV 

WELDING 

Flaming 

48 

60 

0 

60 

18 

60 

0 

60 

Welding 

0 

15 

0 

15 

15 

15 

Cutting  and 
Grinding 

0 

24 

0 

24 

0 

24 

0 

24 

Nuisances 

0 

57 

0 

57 

0 

57 

FIRE 

PDSMOKE 

FIRE_FOV 

WELDING 


Test  Scenarios 

Figure  5-1 1  -  UV/IR  configuration  algorithm  results  by  test  scenario  type 
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Figure  5-12  -  UY/IR  configuration  response  results  by  test  scenario  class 


Table  5-8  -  UV/IR  configuration  response  results  by  test  scenario  class 


New  UV/MIR  Optimization 

FLAME 

SMOKE 

WELDING 

RESPOND 

Flaming  Tests 

48 

0 

0 

48 

Welding  Tests 

1 

0 

15 

0 

Cutting  and  Grinding  Tests 

0 

0 

0 

0 

Nuisance  Tests 

3 

0 

0 

3 

Table  5-9  -  K/UV/IR  configuration  algorithm  results 


FIRE 

PD  SMOKE 

FIREFOV 

WELDING 

Flaming 

49 

60 

0 

60 

18 

60 

0 

60 

Welding 

0 

15 

0 

15 

0 

15 

15 

15 

Cutting  and 
Grinding 

0 

24 

0 

24 

0 

24 

Nuisances 

0 

57 

0 

57 

0 

57 

0 

57 

Test  Case  Alarms  Test  Case  Alarms 
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Figure  5-13  -  K/UV/IR  configuration  algorithm  results  by  test  scenario  type 
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Figure  5-14  -  K/UV/IR  configuration  response  results  by  test  scenario  class 
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Table  5-10  -  K/UV/IR  configuration  response  results  by  test  scenario  class 


New  Opt.  -  K,  MIR,  UV 

FLAME 

SMOKE 

WELDING 

RESPOND 

Flaming  Tests 

49 

0 

0 

49 

Welding  Tests 

0 

0 

15 

0 

Cutting  and  Grinding  Tests 

8 

0 

0 

8 

Nuisance  Tests 

0 

0 

0 

0 

Table  5-1 1  -  NIR/UV/IR  configuration  algorithm  results 


FIRE 

PDSMOKE 

FIREFOV 

WELDING 

Flaming 

57 

60 

0 

60 

18 

60 

0 

60 

Welding 

0 

15 

0 

15 

0 

15 

15 

15 

Cutting  and 
Grinding 

0 

24 

0 

24 

0 

24 

Nuisances 

0 

57 

0 

57 

0 

57 

0 

57 

FIRE 

PDSMOKE 

FIRE_FOV 

WELDING 


Test  Scenarios 

Figure  5-15  -  NIR/UV/IR  configuration  algorithm  results  by  test  scenario  type 


22 


Steinhurst  and  Owrutskv 


FLAME 

SMOKE 

WELDING 

RESPOND 


Test  Scenarios 

Figure  5-16  -  NIR/UV/IR  configuration  response  results  by  test  scenario  class 


Table  5-12  -  NIR/UV/IR  configuration  response  results  by  test  scenario  class 


New  Opt.  -  NIR,  MIR,  UV 

FLAME 

SMOKE 

WELDING 

RESPOND 

Flaming  Tests 

57 

0 

0 

57 

Welding  Tests 

0 

0 

15 

0 

Cutting  and  Grinding  Tests 

9 

0 

0 

9 

Nuisance  Tests 

0 

0 

0 

0 

Table  5-13  -  K/NIR/UV/IR,  Small  SumN  configuration  algorithm  results 


FIRE 

PD  SMOKE 

FIREFOV 

WELDING 

Flaming 

59 

60 

0 

60 

18  X^ 

60 

0 

60 

Welding 

X 

X 

0 

15 

15  X^ 
X^  15 

Cutting  and 
Grinding 

■ 

X 

0 

^X^  24 

0 

^X^  24 

Nuisances 

X 

X 

0 

X^  57 

0 

57 

Test  Case  Alarms  hh  Test  Case  Alarms 


SB  VS  Post-VS4  Algorithm  Development 


23 
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igure  5-17  -  K/NIR/UV/IR,  Small  SumN  algorithm  results  by  test  scenario  type 
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Figure  5-18  -  K/NIR/UV/IR,  Small  SumN  response  results  by  test  scenario  class 
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Table  5-14  -  K/NIR/UV/IR,  Small  SumN  response  results  by  test  scenario  class 


New  Opt.  4  Element  -  Small  SumN 

FLAME 

SMOKE 

WELDING 

RESPOND 

Flaming  Tests 

59 

0 

0 

59 

Welding  Tests 

0 

0 

15 

0 

Cutting  and  Grinding  Tests 

10 

0 

0 

10 

Nuisance  Tests 

1 

0 

0 

1 

Table  5-15  -  K/NIR/UV/IR,  Large  SumN  algorithm  results 


FIRE 

PDSMOKE 

FIREFOV 

WELDING 

Flaming 

58 

60 

0 

60 

18 

60 

0 

60 

Welding 

0 

15 

0 

15 

0 

15 

15 

15 

Cutting  and 
Grinding 

0 

24 

0 

24 

0 

24 

Nuisances 

0 

57 

0 

57 

0 

57 

0 

57 

FIRE 

PDSMOKE 

FIRE_FOV 

WELDING 


Test  Scenarios 

Figure  5-19  -  K/NIR/UV/IR,  Large  SumN  algorithm  results  by  test  scenario  type 
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Figure  5-20  -  K/NIR/UV/IR,  Large  SumN  response  results  by  test  scenario  class 


Table  5-16  -  K/NIR/UV/IR,  Large  SumN  response  results  by  test  scenario  class 


New  Opt.  4  Element  Large  SumN 

FLAME 

SMOKE 

WELDING 

RESPOND 

Flaming  Tests 

58 

0 

0 

58 

Welding  Tests 

0 

0 

15 

0 

Cutting  and  Grinding  Tests 

9 

0 

0 

9 

Nuisance  Tests 

0 

0 

0 

0 

5.3.5.  WELDING  EVENTS 

Review  of  any  of  the  optimizations  in  the  previous  section  shows  that  it  is  possible  to  achieve  a  Pd  of 
100%  for  welding  test  scenarios  with  a  Pfa  of  0%.  The  VSP  data  fusion  algorithms  [11]  rely  heavily  on 
the  SB  VS  WELDING  EVENT  to  guard  against  false  alarms  generated  by  the  other  components, 
including  the  other  SBVS  EVENTS. 
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6.  RESULTS  AND  RECOMMENDATIONS 

As  discussed  in  Section  5,  it  is  possible  to  retrospectively  achieve  better  performance  for  the  VS4  data 
subset  used  in  this  report  than  was  originally  demonstrated  during  the  live  demonstration.  Additionally,  it 
is  possible  to  recover  better  performance  retrospectively  with  fewer  sensor  elements  than  was  achieved 
during  the  original  demonstration. 

The  new  optimization  of  the  UV/IR  configuration  shows  improved  performance  over  the  as-tested 
configuration  and  the  original  2005  optimization.  The  Pd  of  the  FIRE  EVENT  was  improved  from  0.68  to 
0.80  through  optimization,  while  reducing  the  number  of  false  positives  from  7  to  4.  The  Pd  of  the 
WELDING  EVENT  was  also  improved  from  0.93  to  1.00  through  optimization.  The  FIRE  FOV 
EVENT,  which  did  not  alarm  for  any  of  the  selected  test  scenarios  during  the  live  demonstration  and 
alarmed  for  14  of  the  60  test  scenarios  in  the  2005  optimization,  was  further  improved  to  18  out  of  60 
detections.  As  can  be  seen  from  Figure  5-10,  the  features  in  the  algorithm  parameter  space  are  not  shaip 
features  and  the  algorithm  parameter  thresholds  are  not  placed  in  regions  of  rapid  change.  This  makes  the 
response  of  the  algorithms  relatively  stable  with  respect  to  issues  such  as  inter-sensor  calibration.  As  an 
example,  consider  the  discussion  in  the  previous  section  regarding  the  two  possible  4-element 
configurations.  Increasing  the  threshold  value  for  a  key  parameter,  SumN,  by  41%  (from  0.17  to  0.24) 
changes  the  Pd  for  the  FIRE  EVENT  from  0.98  to  0.95  and  the  Pfa  from  0.09  to  0. 1 1 . 

Adding  additional  sensor  elements  to  the  configuration  leads  to  increased  Pd  for  the  FIRE  EVENT  as 
shown  in  Figure  6-1  with  increasing  Pfa  (see  Section  5.3).  The  K/UV/IR  combination  offers  little 
improvement  in  the  FIRE  Event  Pd  (0.82  vs.  0.80)  while  doubling  the  number  of  false  alarms.  This 
combination  is  not  recommended.  The  NIR/UV/IR  combination  offers  significant  improvement  in  the 
FIRE  EVENT  Pd  (0.95  vs.  0.80)  while  only  adding  5  additional  false  alarms  over  the  UV/IR  combination. 
Especially  interesting  is  that  the  FIRE  EVENT  false  alarms  generated  are  moved  from  the  nuisance  and 
welding  test  scenario  classes  to  the  cutting  and  grinding  class.  Hot  work  of  these  types  is  not  typically 
conducted  without  significant  preparation  and  system-wide  notification  in  a  shipboard  environment  and 
any  DCA  systems  would  most  likely  be  secured  or  operating  in  a  special  mode  to  handle  this  type  of 
work.  This  reduces  the  potential  severity  of  the  generated  false  alarms.  Using  all  four  elements  achieves 
an  almost  perfect  Pd  for  the  FIRE  EVENT  (0.98  or  0.97  vs  0.80)  while  adding  only  5-7  additional  false 
alarms,  depending  on  the  threshold  setting  for  the  SumN  parameter.  In  the  case  of  the  smaller  SumN 
threshold,  one  of  the  new  false  alarms  is  from  the  nuisance  test  scenario  class  while  all  others  are  from  the 
cutting  and  grinding  test  scenario  class.  There  are  trade-offs  between  sensitivity  and  selectivity.  From 
the  data  fusion  perspective,  data  from  other  sources  can  be  used  to  tailor  overall  system  performance. 

An  example  of  where  inter-component  data  fusion  might  successfully  be  used  in  the  VSP  is  with  the 
LWVD  Component,  which  takes  a  long-pass  fdtered  black  and  white  video  image  and  generates  a  single 
luminosity  value  for  the  image.  The  long-pass  filter’s  design  cut-off  wavelength  is  720  nm  which  is 
closely  matched  to  center  wavelength  of  the  interference  filter  on  the  NIR  PD  sensor  element.  One  could 
potentially  use  the  LWVD  luminosity  data  stream  in  lieu  of  the  NIR  PD  data  stream  to  produce  an 
effective  3-component  SBVS  system  with  only  the  UV  and  IR  sensor  elements.  Or  an  effective  4- 
component  system  could  be  produced  with  only  the  K,  UV,  and  IR  sensor  elements.  This  would  require 
that  the  LWVD  data  stream  have  similar  sensitivity  and  dynamic  range  to  that  of  the  photodiode.  Testing 
along  these  lines  is  currently  ongoing. 

On  the  basis  of  these  analyses,  the  current  recommendation  for  the  SBVS  Component  in  the  AiOVS  being 
developed  by  VMI  is  either  a  UV/IR  or  a  NIR/UV/IR  configuration,  depending  on  the  data  fusion 
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systems’  tolerance  for  increased  Pfa  with  increased  Pd.  The  EVENT  parameters  for  the  four  potential 
configurations  discussed  here  are  given  in  Appendix  C. 


Figure  6-1  -  Flaming  test  scenario  results  for  the  FIRE  EVENT  for  the 
potential  SB  VS  configurations  discussed  in  the  text.  The  outline  bars  indicate 
the  maximum  number  of  alarms  possible  (60). 
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APPENDIX  A.  SB  VS  EVENT  ALGORITHM  DEFINITIONS  AND  PARAMETERS 

The  finalized  SB  VS  EVENT  criteria  and  parameters  as  used  in  the  VS5  and  DD(X)  Test  Series  were: 

Event:  If  (Sum  N  >=  0.0825)  or  (Abs(5900A)  >=  0.015)  Then 

EVENT  =  TRUE 

Else 

EVENT  =  FALSE. 

Smoke:  If  Abs(5900A)  >=  0.043  Then 

PDSMOKE  =  TRUE 

Else 

PDSMOKE  =  FALSE. 

Fire:  IF  (Sum  N  >=  0.0825)  and  (7665A  >=  0.015)  and 

(10500A  >=  0.015)  and  (RefIR/UV  >=  1)  Then 

FIRE  =  TRUE 

Else 

FIRE  =  FALSE. 

Fire  FOV :  IF  (Sum  N  >=  0.6)  and  (7665A  >=  0.015)  and 

(10500A  >=  0.015)  and  (RefIR  >=  0.2)  and  (UV  >=  0.001)  Then 

FIRE_FOV  =  TRUE 

Else 

FIRE_FOV  =  FALSE. 

Welding:  IF  (Sum  N  >=  0.0825)  and  (7665A  >=  0.015)  and  (10500A  >=  0.015) 

and  (RefIR  <  0.056)  and  (UV  >=  0.175)  Then 

WELDING  =  TRUE 

Else 

WELDING  =  FALSE. 

Persistence : 

IF  EVENT_TYPE  =  TRUE 

EVENT_TYPE . IndexCount  =  EVENT_TYPE . IndexCount  +  1 
IF  EVENTJTYPE. IndexCount  >  51 

EVENTJTYPE . IndexCount  =  51 

ELSE 

EVENTJTYPE. IndexCount  =  EVENTJTYPE . IndexCount  -  1 
IF  EVENTJTYPE . IndexCount  <  0 

EVENTJTYPE . IndexCount  =  0 
IF  EVENT  TYPE . IndexCount  >=  Persistence1 
EVENTJTYPE. ALARM  =  TRUE 

ELSE 

EVENT  TYPE. ALARM  =  FALSE. 


1  For  the  PDSMOKE  EVENT,  the  persistence  value  is  25  seconds,  not  5. 


SB  VS  Post-VS4  Algorithm  Development 


33 


APPENDIX  B.  SB  VS  SENSOR  SUITE  SCALE  FACTORS 

Each  of  the  nine  SB  VS  Component  Prototype  sensor  suites  and  the  AiOVS  mock-up  have  individual 
scale  factors  for  calibration  and  inter-unit  uniformity  in  response.  The  factors  for  the  existing  units  are 
listed  below  in  Table  B- 1 .  There  are  four  generations  of  units:  the  original  units  (S/N  51  -  53),  the  second 
generation  (S/N  54  and  55),  the  third  generation  (S/N  56  -  59),  and  the  AiOVS  mockup  (S/N  21).  The 
RefIRl  scale  factors  for  units  S/N  51,  52,  and  53  were  updated  for  the  DD(X)  Test  Series  conducted 
during  January,  2005  on  the  ex-USS  Shadwell. 
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Table  B-l  -  SB  VS  sensor  suite  scale  factors  by  unit  serial  number  (S/N) 


Unit  S/N 

52 

53a 

Unit  Type 

VSP 

VSP 

NaPD 

KPD 

NIRPD 

ReflRl 

ReflR2 

UV 

255 

255 

255 

DD(X) 

ReflRl 

5.0380 

3.7000 

3.0710 

a  Unit  S/N  53  has  a  split  interference  filter  for 
RefIR2  (2.7  +  4.3  pm) 


Unit  S/N 

54 

55 

56 

Unit  Type 

NaPD 

0.1050 

0.1050 

0.2600 

KPD 

0.0038 

0.0038 

0.0470 

NIRPD 

0.2530 

0.2530 

0.2530 

ReflRl 

2.1000 

2.1000 

0.5250 

ReflR2 

3.2500 

3.2500 

3.2500 

UV 

255 

255 

255 

Unit  S/N 

57 

58 

59 

Unit  Type 

VSP 

VSP 

VSP 

NaPD 

KPD 

NIRPD 

ReflRl 

ReflR2 

UV 

255 

255 

255 

Unit  S/N 

21 

Unit  Type 

AiOVS 

NaPD 

0.25 

KPD 

0.05 

NIRPD 

0.25 

ReflRl 

0.3100 

ReflR2 

3.2500 

UV 

255 
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APPENDIX  C.  SB  VS  EVENT  PARAMETERS  BY  CONFIGURATION 

For  each  of  the  four  AiOVS  configurations  discussed  in  this  report,  the  complete  listing  of  SB  VS 
algorithm  parameters  used  in  the  analyses  are  given  in  Table  C-l. 


Table  C-l  -  Potential  AiOVS  configuration  SVBS  algorithm  parameter  values 


UV/IR 

K/UV/IR 

NIR/UV/IR 

K/NIR/UV/IR 
Small  SumN 

K/NIR/UV/IR 
Large  SumN 

EVENT 

Sum  N 

0.001 

0.001 

0.001 

0.001 

0.001 

Na 

FIRE 

Sum  N 

0.0017 

0.0017 

0.025 

0.017 

0.024 

K 

0.03 

0.0007 

0.0007 

NIR 

0.01 

0.01 

0.01 

ReflR/UV 

0.5 

0.5 

0.5 

1 

1 

PDSMOKE 

Na 

FIREFOV 

Sum  N 

0.006 

0.006 

0.006 

0.006 

0.006 

K 

0.015 

0.015 

0.015 

NIR 

0.015 

0.015 

0.015 

UV 

0.001 

0.001 

0.001 

0.001 

0.001 

Ref  IR/UV 

0.08 

0.08 

0.08 

0.08 

0.08 

WELDING 

Sum  N 

0.0825 

0.0825 

0.0825 

0.0825 

0.0825 

K 

0.015 

0.015 

0.015 

NIR 

0.01 

0.01 

0.01 

UV 

0.3 

0.3 

0.3 

0.3 

0.3 

RefIR 

0.005 

0.005 

0.005 

0.005 

0.005 

